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1. Introduction 


This constitutes the final report on the research being performed 
by the School of Aeronautics and Astronautics, Purdue University, for 
the NASA Langley Research Center under grant number NAG-1-305. The 
area of research is niulti-input/multi-output control synthesis techni- 
ques nrativated by applications such as forward-swept-wing aircraft, 
which exhibit significant rigid-body/aeroelastic modal coupling. 

2 • Cownent on Personnel 

A major portion of this research was performed by a doctoral 
graduate student (Mr. Mike Gilbert), while the student v/as in resi- 
dence at tne Langley Research Center. This was an important element 
of this program, providing the opportunity for this work to actually 
proceed "in situ" with the graduate researcher interacting daily with 
NASA's international experts on control of aeroelastic phenomena. 

In addition to the research reported herein, the graduate student 
actively participated in experimental evaluations of several candidate 
control laws for stabilizing a dynamically scaled, statically unstable 
wind tunnel model of a forward-swept-wing aircraft tested in 
Langley's transonic dynamics test facility (TDT). 

3. Suimiary 

In the early phase of this research, the potential of cooperative 
game theory for multi-variable control synthesis was briefly explored, 
and a summary of key concepts are attached as Appendix A to this report. 
The key conclusion of this survey was that if a multivariable (multi- 
input/niulti-output) control law is to be synthesized to be cooperative. 
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this corresponds to a Pareto-optimal rather than a Nash solution to the 
mathematical game, and the Pareto solution may be found via Linear 
Quadratic Regulator control theory. On the other hand, if, for example, 
more than one interacting controller is not being synthesized by the 
same "designer," the optimal solution is the Nash equilibrium. 

A second method of modern control law design was proposed that 
addresses the problems of selecting the cost functional which the 
control law is to optimize, and the lack of other useful design infor- 
mation. The approach recognizes that in any optimal control problem, 
there are many more design parameters to be selected than the gains of 
the optimal control law. These additonal design parameters may be part 
of the system dynamics or they may be part of the optimal control 
problem formulation. Either way, by calculating the sensitivity of 
the dynamical system time and frequency domain performance to these 
parameters, systematic ways of altering these parameters to improve 
the performance of the control law can be developed. For example, if 
the parameters are selected as the nominal values of the poles of a 
compensator, the compensator design can be adjusted to provide better 
performi’nce. Similarly, if the parameters are entries of the optimal 
control problem cost functional, then the control problem itself can 
be altered to meet time domain design specifications. This technique 
titled the "Optimum Parameter Sensitivity" approach is presented in 
Appendix B of this report. 
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Aspects of Non-Zero Sum Differential 

Game Theory with Application to Multivariable Control Synthesis 

M.G. Gilbert 
April 11, 1983 

Introduction : 

Modern, optimal control theory and differential game theory (1-4) 
were developed concurrently but independently during the early 1960’s. 

Both are concerned with obtaining optimal control strategies (open or 
closed loop) for n.ulti-input systems describable by a set of differential 
equations. Optimal control problems are characterized by a single controller 
using a control law picked to minimize a single scaler objective function. 
Non-zero sum differential games on the other hand involve several players 
(controllers) each attempting to control the system to minimize it's own 
cost function in the presence of the control actions of the other players. 
Thus differential games are characterized by multiple controllers minimizing 
multiple cost functions, leading to a ve ctor minimization problem in 
order to obtain optimal control strategies. 

Modern optimal control theory has been studied extensively and has 
found application in developing control laws for state regulation, terminal 
guidance, and process control. Much less is known about differential 
games, whose primary applications have included pursuit-evasion studies 
and economics problems. It is generally recognized that optimal control 
can be viewed as a subset of differential game theory, as will be apparent 
in the linear, time invariant, game to be discussed in the next section. 
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Some Aspects of Differential Game Theory : 

Generally, the average control law designer is much more familiar 
with optimal control theory than differential game theory; for that reason 
this section will highlight some important features of differential game 
theory. The discussion which follows pertains particularly to linear, 
time invariant, non-zero sum differential games which can be modeled 
mathematically as 


m 

X = Ax + y B.u. 1 ) 

i=l ’ ’ 


It is assumed that each player or controller picks his control u^. to 
minimize a quadratic cost function of the form 




T 

{x'Q X + I 
' j=l 


'R..u.)dt 
iJ J 


2 ) 


Note that the ith player may be penalized for the player's use of 
control energy. 

A 

In general, it cannot be expected that a control strategy set U = 

{u-j, ..., u^} can be found which will absolutely minimize every player's 
cost function in the presence of the other player's control actions. If 
such a control set exists, it could be found by solving m uncoupled 
optimal control problems with each player independently controlling the 
system. Because such a control set cannot usually be found, definitions 
of the sense in which a control set U* is optimal with regard to another 
control set U are needed. There are three widely accepted definitions 
of when a particular control law solution set U* is optimal in differential 
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game theory. These are the Nash Equilibrium solution, the min max solution, 
and the Pareto-optimal or non-inferior solution. 

Of the three solutions, the Nash Equilibrium and Pareto-optimal are 
the most useful for the development of control laws satisfying the game. 

This is because the min max solution assumes irrational behavior of the 
other m-1 players when solving for the ith players control law. Specifi- 
cally, the solution assumes the m-1 other players are ignoring their own 
costs and using their control to maximize the ith players cost. The game 
then becomes zero sum (the ith player minimizing his cost and the m-1 
players combined maximizing it). The solution is overly pessimistic 
and may fail to be finite for the ith player even in well posed games. 

Alternatively, the Nash Equilibrium solution assumes rational beha- 
vior of the players. The ith player’s Nash Equilibrium control minimizes 
his cost function when the other m-1 players play their own Nash, controls. 
Mathematically, this is stated as 


J,(u, 


'i-r ""i* “i+r 




) ’) 


★ * 

where J,j is given by 2, u-j , . . . u^^ are the components of the Nash control 
set e (u.| , .. ). and u,j is any control strategy other than the Nash 

control for the i^^ player. The Nash solution is a stable equilibrium 
solution, since a player cannot deviate from his Nash control without 
incurring increased cost if all the other players use their ov/n Nash controls. 

Necessary conditions for the Nash Equilibrium solution for the game 
given by equations 1) and 2) are 


★ -IT 
Ui =-r^!b/s.x 


4 ) 


where S. are solutions of the m coupled Ricatti like equations 


S, = -S,A - A^S, - q, . 


5 ) 


■1„ T, 


. s,(t,)=0 


Unfortunately, these equations are difficult to solve and only a few 
sufficient conditions for the existence of solutions to the equations are 
known (Ref. 5). In the single player case (m=l), the equations 5) reduce 
to the single Ricatti equation of optimal control theory. 

The Nash solution, stated mathematically by 3), is an uncooperative 
solution since each player is only concerned with minimizing his own cost, 
and cares nothing about the other player's costs. The question then 
becomes whether or not other solutions exist which simultaneously reduce 
the costs of the players from their Nash costs. Pareto-optimal or non- 
inferior solutions to the differential game may be solutions with that 
property. In fact, Pareto-optimal solutions have the property that 
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or there is at least one i=l, ... m such that 
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where are the components of the Pareto-optimal control set 

.... u^*), and u-j , are controls other than the Pareto- 

optimal controls. There are many solutions satisfying 6A), 6B), some 
of which may have costs for each player lower than their corresponding 
Nash costs. Selection of a particular non-inferior solution in all cases 
involves trading off the costs of one player over another, thus the 
players are faced with a negotiation in order to obtain a solution, or 
else some level of cooperation among players must be enforced. 

It has been shown (6, 7) that some* of the non-inferior solutions 
to the vector optimization problem can be obtained by solving an m-1 
parameter family of optimal control problems, if the cost functions 
satisfy certain convexity requirements. Mathematically, this is stated 
as follows. The control set i^*(u^*, ..., u^^^*) Is Pareto-optimal (non- 
inferior) if 


< J(’i') 'V ^ <}>* 


where = t , a. > 0 V-i=l, ..., m, and t a.= 1 

i=l 1 K 1 


J(l|;*) < Ij- ij)* 


where \ , a. > 0 V- i = 1 , . . . , m, and ) a. ~ i. 

i=l ^ ^ i=l ’ 


*For the linear, quadratic cost game, it is believed all could be found, 
however unrealistic it may be to do so. 
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By 7A, the sets i’* which give the unique minimum of any individual 
is Pareto-optimal. Note that selection of a set of thereby defining 
a particular solution, is equivalent to solving the negotiating problem 
referred to previously. 

For the linear, quadratic cost games of equations 1) and 2) (the 
of 2) satisfy the necessary convexity requirements, see Reference 8), 
the Pareto-optimal solutions are 


★ r 1 T 

U. (a) = - I a.R. B.'s(a)x 
1 [j = l J 1 


where S(a) is given by 


S = -SA 


• *1 

T m m ri -1 

AS- I a.Q. + S B. i a.R.. B. 
• 1=1 ’ ^ i = l ’ [j-1 J ’ 


9) 


S(t^,a) = 0 


with the a. satisfying the conditions of 7A), i.e. a. > 0, 

There is a geometrical interpretation, first given by (3), which 
is useful in understanding the Nash and Pareto-optimal solutions. Con- 
sider a two player game, with contours of constant cost plotted in the 
U.J, U 2 control space, as shown in Figure 1. 

The short-dash lines are the minimizing control for one player 
when the other plays every other control in his admissable control range. 
The intersection of the two dash lines is the Nash Equilibrium solution, 
which occurs at the intersection of two cost contours. The cross hatched 
area represents admissable controls u^ and U 2 , which if played would 
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Figure 1 

result in reduced cost (from the Nash cost) for both players. The long- 
dash line, which follows the tangent points of the cost contours, are the 
Pareto-optimal solutions. These are the minimal cost solutions for each 
player. They require cooperation, since both players must trust the 
other not to play a minimizing control in response to the first's use 
of control in the shaded area, and therefore these solutions are vulner- 
able to cheating. For example, say player two uses a control on the 
tangency line in the cross-hatch region in anticipation of player one 


using his control to arrive at solution A, but player one cheats, playing 
his minimizing control resulting in solution B. This is the so called 





linear games with quadratic costs, have. Note too what would happen if 
the Nash solution were also non-inferior; the non-inverior solution 
would be stable and invulnerable to cheating. This desirable situation 
does not occur in general. 


Appl ic a tion to Control Law Development : 

The last section discussed the types of solutions v;hir occur in 
differential games in general, and- linear, time invariant, quadratic 
cost games in particular. There are many more important properties of 
these solutions which were not discussed. Rather than devote much more 
time to these important properties, the potential of differential game 
theory to improve control law development for multi-input linear systems 
will be discussed. 

Optimal control law synthesis for multi-input systems has in the 
past been primarily performed using modern optimal control theory, that 
is by minimizing a single scaler-valued quadratic cost function. This is 
because modern control theory takes advantage of powerful matrix l■'.2thods 
for algaebraic manipulation, because most control system designs have 
in mind a single overall objective, and because of the difficulty of 
obtaining solutions to vector-valued optimization problems. For example, 
primary aircraft flight control systems have as their main objective 
improvement of the aircraft handling qualities. In those cases where 
several systems were desired, say an automatic flight control system 
and an active structural control system, the nature of the system plant 
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(aircraft) was such that the systems might be designed separately. These 
are cases where a control set uniquely minimized both cost functions. 

With the increasing complexity of aircraft design has come instances 
where several control systems could not be designed separately, the 
Grumman forward-swept-wing (FSW) demonstrator being an example. Because 
of the coupled rigid-body and structural dynamics of the FSW configur- 
ation, the separate flight and structural control systems of that 
aircraft have interacted unfavorably with each other. This is a case 
where an integrated approach, satisfying two different, broad, overall 
objectives and making use of differential game theory solutions might 
lead to better control laws for both systems. 

Of course the question remains as to what is meant by better? All 
differential game theory has promised is that the individual cost 
functions, which may or may not have physical significance, will be 
minimized taking into account the control action of the other. The 
important physical properties of these solutions, like closed-loop 
eigenvalue locations, robustness properties, frequency response, etc., 
remain unknown. On the plus side h. wever, there are now more solutions 
(control laws) to evaluate which at least have known properties among 
several scaler parameters than there are with an optimal control solution. 

There exists at least one example where a Nash equilibrium solution 
(while n_ot explicitly labeled as such) was proposed for control system 
development (9). The methodology attempted to account for the control 
action of the aircraft pilot in the design of an aircraft longitudinal 


stability augmentation system. The pilot was assumed to be an optimal 
regulator, leading to a two player, linear, quadratic cost differential 
game. Properties of the solution like eigenvalue location were not 
investigated as the scaler cost of the pilot was directly related to 
the Cooper-Harper handling qualities rating scale, the primary objec- 
tive of the methodology being the best (lowest) Cooper-Harper rating 
possible. This example does serve however as an incentive to investi- 
gate the potential of differential game theory for control law design, 
since it was advantageous to use the theory in this case. 

Questions : 

The previous section has vaguely called for an "investigation" of 
differential game theory in the context of multi-input control system 
development. Some work is known to have been done in this area (10). 

The author is puzzled though that apparently a lot more has not been 
done. Either there exists unknown work which dispels any advantage of 
game theory in these situations, or else they have never been considered 
due to their complexity, the lack of a need to integrate several systems, 
or whate'er. If the latter is the case, then enough questions arise 
about the differential games solutions and their applications to advise 
many investigations. Some questions are listed here. 

1) Only a few sufficiency conditions for the existence of Nash 
solutions are known, given in terms of the norms of defined 
matrices when either a) the system A matrix has a prescribed 
degree of stability, or b) the solution to certain auxiliary 


u 


control problems exist (5). Are there conditions on the 
weighting matrices of the cost functions (like observability 
and controllability concepts from optimal control) which 
determine directly if a Hash solution exists, rather than 
through the solution of an auxiliary problem? 

2) Are there weighting matrices in the cost functions such that 
the Hash solution is also non-inferior? Such a solution would 
have many desirable properties. In addition, the mathematical 
question of when a set of coupled Ricatti like equations could 
be solved by a single Ricatti equation would be answered. 

3) Are there enough relationships between the individual cost 
functions and the physical properties of the closed-loop system, 
i.e. eigenvalue locations, frequency response, etc., to justify 
considering Pareto-optimal solutions over the Nash solutions? 

If so, can the problem be further reduced to a simple modern 
control problem with pre-defined structure? Or is the Pareto- 
optimal solution just another way of picking weighting matrices 
in modern control problem cost functions, similiar to methods 
used in Bryson and Ho and Kwakernaak and Si van? 

4) What kind of solution is a linear, quadratic cost, multi-input 
modern control problem in the context of differential games? 

Are the individual controls in the problem in a Nash Equilibrium, 
are they cooperating in a Pareto-optimal sense, or are they 
in a "prisoners dilemma" situation? 
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OP r I HAL LINLAK CONTROL LAW DESIGN USING OPTIMUM 


PARAMETER SENSITIVITY ANALYSIS 


Michael G. Gilbert 


A Proposal Tor Doctoral Dissertation Research 
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I - INTHODUCT lUM 


The usG Q-f -feudbcick is a well known and effective means 
of altering the dynamics of a system in order" to improve 
stability, reduce sensitivity to model errors, and meet 
per f or mance specifications. Tine design methods used to 
develop these closed-loop control laws can bo broadly 
classified into two categories, classical and modern. 
ClCHSSical methods provide systematic design information that 
is used by the designer to develop single loop control 1 er's , 
possibly with their own dynamics, to meet performance 
specifications. They ar'e less than easy to use if the 
system is mul t i ■ i np'..(t /mul t i -output in nature. Modern 

methods, on the other hand, determine high order control 
laws easily, and they optimize a quadratic functional of the 
system states. Tlie disadvantage of modern methods is the 
difficulty of writing the cost functional to reflect the 
performance specifications of the system, and the lack of 
systematic redesign information if the original design is 
un s a t i sf ac t or y . 


A method of modern 

control 

1 aw 

design fia 

s 

been 

devel oped 

which addr'e^jses 

the p)roblems 

of writing 

t h e 

cost 

f uncti onal 

which the control 

1 aw i s 

to 

op t i m i z e , 

and 

the 
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1.AC.I-; of UjC-ful d-j-itQn i fi-f or mdt i on . The method recoqnx::es 
t^tat in any optimiil ccintrol problem, there are many more 

design parameters to be selected than the gains of the 
optimal control law. These additional design paronieters may 
be part of the system dynamics or they may be part of the 
optimal control problem formulation. Either way, by 
calculating the sensitivity of the dynamical system time 
dontain performance to these parameters, systematic ways of 
altering tht?se parameters to improve the performance of the 
control law can be developed. For example, if the 
parameters are selected as the nominal values of the poles 
of a compensator, the conipensator design can be adjusted to 
provide better performance. Similarly, if the parameters 
are entries of the optimal control problem cost functional, 
then the control problem itself can be altered to meet time 
domain design specifications. 

I. a - Background 

The concept of sensitivity of optimum solutions to 
problem parameters has recently been developed by Sobieski, 
Barthelemy, and Riley C13, and Barthelemy and Sobieski [2,33 
in the field of parameter optimization theory. This concept 

differs from the traditional idea of sensitivity analyses in 

* 

that the results etre used in the design process rather than 
ttie performance iisseostneriL stage. What this means is that 


/ 
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hlic? sensitivity diita is used to redefine the original 
optimization problem so as to improve performance by 
altering preselected parameters of the problem (parameters 
not selected by the optimization). Prior to this 
development, sensitivity analyses were used to determine 
performance changes to arbitrary, uncontrol lable 
per turbati ons in system parameters. The optimal sensitivity 
derivatives calculated by Sobiesf:i, Barthelemy, and Pi ley 
were obtained by di f f erenti ating the necessary conditions of 
optimality for non-linear programming problems with 
inequality constraints. One of the most interesting 
applications of the optimal sensitivity results has been in 
the development of multi-level optimization schemes for 
large structure parameter optimization problems (Sobieski, 
James, and Dovi C43). 

Sensitivity analyses are commonly used in both 
classical and modern control theories to assess the effects 
uncontrolcd variations in the system parameters will have on 
system per f or<nance , primarily in stability margins. Methods 
have been developed by various researchers <e,g. Yedaval 1 i 
and flkelton C53) for designing control systems which are 
insensitive to parameter variations. Only recently, time 
response sens i t i v i t >' was introduced as a means of assessing 
S'/atem performance changes to parameter variations by 
Gclia.ecltter [63. Neithc?r of these app 1 i c at i orts of 
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•jei iiii 1 1 VI ty u-^csj thtj j n f or <not i on at; part of the design 
process however- There also e;:ist several methods for 
defining the moder o control theory problem so as to meet 
certain system objectives, besides the commonly used 
intuition of the designer. These include selecting 

weighting matrices on the basis of asymptotic properties of 
linear regulators (Harvey and Stein C73 and Stein COD). 
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Other areas of control system design worth mentioning 
are output feedback theory, reduced order compensator 
design, and frequency shaped linear regulator design. 
Output feedback theory was developed by Levine and Athans 
C9D and solves the problem of feeding back fewer outputs of 
the system than there are states. Reduced order compensator 
design is receiving attention because it recognizes the 
actual control law structure that is usually implemented on 
working hcirdware. One method of designing reduced order 
compensators has been parameter optimization methods 
(Muk.hopcAdhyav' , Newsom, and Abel CIOD). Frequency shaping 
techniques of linear reguJ ator design were developed by 
Gupta C113, by making the weighting matrices of tlie optimal 
control problem functions of frequency. These areas of 
control theory are noted because they have the potential to 

make use of optimal sensitivity analyses as will be 

» 

presented in ttiis p^^po^. 
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I hu pr Qpu‘_it?d cii-t-fiod of cofil.rcjl law dcEiyn (nal.e“t use of 
the idea of optimal sensitivity to problem parameters to 
iles-elop arid use systematic design data to further improve 
controlled system time domain performance. This is to be 
done by computing the sensitis'ity or derivative of the time 
doin;<in resp'snse, either absolute or mean sguare,to 
parameters of the problem not selected as part of the 
op t i m i a 1 1 on pr'ocess. These sensitivities are then to be 
used to redefine the parameters so the problem can be 
resolved, viith an e.ipected improveinent in per' f oritiaiure. The 
sensitivities are computed in a two step process. First, 
the sr.-nsi t i V 1 ty of the necessary condition for the control 
to be optimal is obtained, and then the serisitivity of tlie 
time responrse is computed as a function of the sensitivity 
of the? necessary condition. The result xs a connection 
between the time response of the system and the parameter's 
which specify the optimal control problem being salved. 
With this connection, it is possible to specify changes in 
the designer selected parameters using a first order Taylor 
serioHi, and , the optimal control problem resolved, leading to 
a two stage optimal control design process. 


11 - hl THGDOl <)0i' 
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This section presents a discussion of the use of 
par arnetur sensitivity analysis in the development of linear 
regulator control laws. In this section, the tinie domain 
state, output, and control responses, both absolute and RMS, 
are of interest. Ttie derivation of the sensitivity of these 
time responses to arbitrary parameters of the d/namical 
systeni or to parameters of the control law formulation will 
developed in ttie first subsection, followed by a review of 
linear regulator tfteory. The following subsection will then 
combine the pararneter sensitivity results and lin>*ar 
regulator theory to give systematic niethod for either 

defining the linear regular problem or altering the system 
dynamics to achieve desired time domain responses. This 
section will then be concluded by a discussion of practical 
asf;)ect3 of ttiis metfiodol ogy , as i t applies in the case of 
time domain design criteria. 

II. a - Parameter Sensitivity of Time Domain Responses 

Tlie sensitivity of time domain state, output, and 
contrcjl resporises to arbitrary parameters that appear in the 
fno.lol or conLr'oi law f or mul a t i on , for a linear, tiine 
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invariant. dynamical system is derived in this subsection. 
The derivation begins by considering the state space model 
of the system in the form 
= A;< + E«u + Dv 

(II. n 

y - Ci< 

where :< is an n-dimensional state vector , u is an 

m-dimensional control vector, y is an 1 -dimensi anal output 
vector, and v is a (.-dimensional disturbance vector which 
may be either deterministic or stochastic. The matrices A, 
B, C, and D are constant coefficient matrices of the 
appropriate dimensions. Assuming that a linear, full"State 
feedback control law e::ists and is specified as u = --G!! , 
then the closed"laop dynamics of the system are given by 

= A:, Dv (II.2> 

where A is defined as 

A = A - B6 (II.3> 

The closed-loop system given by eqn. (II. 2) is a 
function of the original model dynamics matri;; A, input 
(natrix B, and ttie gain matri:-: G. Each of these matrices may 
themselves be functions of parameters which may vary due to 
a natural process of the system, may be imprecisely l.nown , 
or may be specified by the designer. For example, the 
entries of tlie A matrii? may be subject to modeling eirors, 
and the entries of the gain matrix 0, if they are obtained 
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U'ui I'lij upti.njil cciiiii'i.j) t.lu:'ory, i^re functions, at dc'Oigmjr 
ac-'lected weightings.. H i3 puesibla to determine tfio 

effoct'j these p -jr am'- h erg have oi1 the state. output, and 
catitrol response of tfie closed- loop system (1 1. 2) in ttio 
following manner . 




5-3 


la 




Tfie time resp'jnse of the closed-loop system <11.2) due 
to the di stnr ban(.<_* '»■ is w'el 1 Inown and is given in terms of 
the stale transition matri;: and matrir super pos i 1 1 cm 

integral [121 as 

f 

( 11 .^ > 


::<t) - '|J<t,t,.) + ) & <t ,T>Dv (•tid'd 


vitiero 5<t, ,t.) IS the state transition matrir of the 


closed- loop dyoamicE matri:; given in eqn. (1 1. 3). In order 

to calculate the siensitivity of the state t'o arbitrary 

par iuneter s , it is a-l course possible to diferentiate eqr.. 

(II. 4) with respect to tlio'se parameters, but because the A 
matri:; of eqn. <11. 3) is in general not symmetric, the 
di f er enti ation will prove difficult. The alternative is to 
<J i f- f e*r ent i ate eqn. (II. 2) first, and then write Die 

super pos i h i un integral f'or that e;;pression, F’roceeding 

dp dp 

where p is the arbitrary pce-ameter, and it has beeri as,su.nied 
tliat t.he di sturbance v ;>nd the way it enters the s.siem 

f 

'thi u'.igti the matri:; D) are independant o< ttie (i ar amet er . 
(lie i imi- r e-.'.pQns'i of C'-in. (II. b) is similiar L'j t.l'O -j/slem 


3‘- J” _d'' 
= -r A- - 


I I . S > 


(he 


I 1 mi • 


of C'-in. 


( 1 1 . b) IS 
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hiitif? r w‘'ipQu>jC‘ in Ui <t l.he transition matrir o-f 13 

identical to t liat of (11.4), with eqn. (Il.S) driven by tfio 
state of the oloiedloop system; 

<f •' ” f TT 

= ^(t,t.)--(t. ) + H<t,'l')--K('Ddr (II. 6) 

■^p Jp J. ^p 

In general, tho initial state of the system will also be 
independent of tf>e parameter p, so that eqn. (II. 6) will 
reduce to tho calculation of the integral term only. 

Tfie above resul ts gis-e the sensitivity of the state to 
protjlem parameters; if the sensitivity of the system outputs 
or controls are of interest, tfio results of eqn. (I 1.6) can 
be used to calculate these quantities as well, since both 
are linear functions of the state. For the system controls, 
the sensitivity to problems parameters is obtained using 
u ~ “G:; as 
u G 

— • = ■- — :: - G— (I I. 7) 

9p 5p Op 

and the sensitivity of the system outputs to tfie parameters 
i s 

5 c 

— ,= — K + C-- ( 1 1 . 8 ) 

9p 9p Bp 

The sensitivity results that tiave thus far been 
obtained are for cases wliere ttic sensitivity of the absolute 
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tunc I 15 c.f i- ,1 . If th,j eovar i of the 

oyttem -..1 ..t e i of iiiioroi.t. i, djf f t,*r ant set of results can 
he oht... MK.-il using tf.e closu<J-loop syste.n covariance 
equations. Tlie covariance r osponce of the system given by 
oqn. (11.2) to the disturbance v Ir. f. 13] 

; . 5X . Kftt „VDtx<t,, . 

Where X is do fined as tfu? system covariance matris:, A is the 
cloiecl-loop dynamics matri.;, and V is the intensity matri;: 
Of V If V IS a crjfo- mean, Gaussian, white noise, random 
vector , or the square disturbance it v is deterministic (if 
V IS det<?t mi 111 St i c , then X is the square response). The 

sensitivity of the syctom covariance to the paramoter p is 
just 


ax 

3p 


_ox 5x 
A — + — A 
5p ^p 


SA 

— X 4 



< 1 1 . 1 1;> ) 


where it has again been assumed that the disturbance and the 


way it enters the system are independent of the param< 5 ters. 
Not ice that eqn . (II. 10) has the same form as eqn. (II. 9), 
and contains the system covariance e;;plicitly. 


It the closed-loop control law yields an asymptotically 
stable system (all the eigenvalues of (II. 2) strictly in the 
lef t-halt-plarie) , tfien the covariance equation (II. 9), tor 

•tie case of a stationary random input v, will achievi? a 
steady -state solLition ClT-3 giv-on by 
o •-= 7'.x > xA^r nvo'^' 







OF POOR 


■«nd t:tu» senii t, i VI ♦. V trqu^tian (II.IO) will ail'-ici hcwcj a 
*,Lt?t«dy atotc c-ulutioti [&3 given by 


„^X iX.T 3 A 

0 = fy.. 4 ^ — X + X — 

d (3 3p 3p 


<11.12) 


The '.cnsitivlty oi the control and output covariances 
are obtained from the state covariance sensitivity in the 
^ollowirig manner. The cofitrol cos'ari ance is, in terms 
th.e state covariance, C133 


U =■- GXG' 


( 1 1 . IV. > 


Di f f erenti ating eqn. (11.13) with respect to the parameter p 


gi ves 

;>U 3G ^ OX T 3G^ 

— = -- XG + G--g‘ GX — 

"Sp 3p ^p 


(11.14) 


The sensitivity ot the output covariance C13D is obtained 
similarly as 


Y = CXC 


(11. lb) 


3y 3C ^ -3X ^ 

= — XC ■*• C---C' + cx — 

3p t)p ^p ^p 


(11.16) 


Il.b - Optimal Linear Regulator Theory 


Tn t.tiis ‘Mita' v.ut: t i on , a. brief review of linear- r-egulatar 


tt leor y 


preoiritecJ fur the? 


i nf 1 n i t«i time, 


const. t,nt 




§ 

•1 

u 

*< 
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c,<3v^^ *' i c 1 <!i . f 1,1 1 1 t c«.A3u. Thia maturi,?! ui I 1 

L)o used 111 tlte rtc;;t •bub^cction along wi tf» sensitivity 
res'ilts to develop a methodology of control law design to 
meet time domain response criteria. 

Consider again the open-loop dynamical system given by 
eqn. (II. 1>. It is desired to find a state feedback control 
law, that IS, a control policy which is a function of the 
states of the system, such that the stc»tes of the system 
•stay close to ceru in the presence of the disturbance v. In 
fact, the objective of the design can be generalised to 
achieve desired output perfoi'mance in the presence of the 
disturbance, by using the output equation in (II. 1). The 
optimal solution for this problem, wfien the objective is 
stated as a quadratic function of the states and controls, 
is the standard 1 i near , quadratic regulator. The LO 
regulator problem is formulated as follows. 


A scaler cost functional of the outputs and the 
controls is formed as 

y 0 ' V' u Ru J d t ( 1 1 . I '/ ) 

where O' and R are designer seltjcted weighting matrices 
which reflect the importance of the responses to the 

designer-. The 0' matri:-; muvit he positvo semi def i ni te , and 

« 

the R inatri;: must be positive definite C143. The cost 



f unt.;f: i onal J givcMi tay can be rewritten in ter- ms of 
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i I'.o H./ 'll '.‘III *il by defining D = C Q'C, 

0 :. ♦ u f<u)dt (IJ.18) 

111 * n:..<? V <:'jnditiuns for the control u to (ninioilzc? the 
vti'jt functiontil of eqri. (II.IB) yield the equations C143 
u «• • r' ( t);: (t) <11. 19a > 

P ^ -PA - A^P + PBr'i/p - Q ; P(t^> >= 0 (II. 19b) 

If the m?itri:: pair' <A,^D) is completely observable C153, 
then tlie matri:' differential Riccati equation given by 
(II. 19b) will have a steady-state solution for P which is 
given by 

<:• -■ • f-'A - (7 P > PE<k' P - Q (II. 20 ) 

In the steady-state case, the optimal control policy becomiss 
time invariant, the gain matri:: defined as G -- D f' is 

constant, and the closed-loop system dynamics is given bv 
eqn. ( I 1 . 2) . 

II.c — Regulator Design Using Parameter Sensitivity 


With the re.=i.ilts of the two previous subs‘?'Ct. i ons , a 
methodology of control law design to meet time oomaj n 
r ijsponse criteria, using an optimal control law, will now be 
presented. This friethoi-J addresses the main drawbaci: of L.Q 
requlato." theor-/, and that is the need for tlae L.Q. 'desigrier 
to sel'act certain desiyn parameter-. wit)t':iut the benefit at a 
pl’y'ii c.-.il rel a t i 'jnr.hi p tj*'twe<?n thaii'.' para.n'ater s and the 
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tjctual Licnu t e'ipon-se^ of the system. For e;:ample, by 

selectiriy el fmctit s cif the O' and R matrices as par amct ors , 
the parameter- sensi ti vi t-/ results of subsection 1 1, a can be 
used to define the optintal LO problem to meet the time 
domain criter la. The fact that a physical relationship 
between the designer selected weights and the time domain 
responses can lie established is seen by extending the 
parameter sensitivity results. 

To begin, recall that both the absolute time response 
sensitivity, eqn. (II. 6), and the covariance sensitivity, 
eqn. (II. 10), are functions of the partial derivative of the 
closed-loop dynamics matrix A with respect to the parameter 
p of interest. For tl\e case of the optimal LQ control law 

the partial derivative can be calculated, using eqn. (I 1.3). 
Thus 

3A ■^A UB 3G 

— = — _ — G E, — (11.21) 

■^p ^p 9p 3p 

Now, the partial derivative of the gain matri;'. G witfi 
respect to the param<|ter p is calculated as 

Sr*' ^ ^ 

— e P + P + p'g' — (11.22) 

An ei^pression for the pdirtial derivative of the steady-state 
matrix Riccati equation (eqn. 11.20) can be obtained b-y 

t 

ariding and s'tbtr act i ncj 2PDf'< B F‘ to eqn. (11.2fi> to get 
O ^ (A - DR~'b''^F’)^ f t F (A - D(r'ti^P) O 1 PBr' b"'^ F’ ai.23) 


# 


i 






'(tp 

[do ?a 


A — 

+ — A 

-- + — p + 

P — 

9p 

*^P 

i3p Sp 



pB .1 T 

5R ' 

.,3Bn 

- pj 

— R B 

+ D — + 

BR — - P 


i^P 

3p 

^p/ 
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l)i f f <;fenti al. \ luj t'qii. (11.23) with r'osp&ct to p, canceling 

-( T 

tfcfr mb , and roplciciny F\ B P by 6 gives 


(11.24) 


With eqn.'s (II. 21), (11.22), and (11.24), it is now 
possible to calculate the sensitivity of the absolute time 
response to parameters of the optimal L(3 regul ator problem 
using eqn. (1 1. 6), and the sensitivity of the state 
covariance to the parameters using eqn. (II. 10). Control 
and output absolute time response sensitivities are 
calculated from eqn. s (II. 7) and (II. 8) once the absolute 
state sensitivity is obtained. Control and output 
covariance sensitivities are obtained from eqn.s (11.14) and 
(11.16) once the state covariance sensitivity is calculated 
from eqn. (II. 10). These sensitivities are tlie physical 
lints between the designer selected parameters of the LQ 
design problem and the actual time responses of the 
closed-loop system. 


To illustrate the use of optimal parameter sensitivity 
analysis in the design of optimal linear regulators, the 

following situation is considered. Assume that a dynamical 

» 

system modeled by a linear' statr? space r epresentr»t i on is to 
be controlled by feeding bact. all the s.ystem states. 



16 


ORiGiU''*' 

OF POOR 

includiny which are used in the model to represent 

ttie dynamics of the control actuators. A quadratic cost 
function of the form given by eqn. (II.lQ) is written, which 
weiglits the states and controls. The optimal linear 
regulator problem is solved (eqn.s 11.19), and the system 
state covariance response to a random input calculated from 
eqn. II. 11. If the RMS response of one or more of the 
states is unacceptable to the designer, sensitivity analysis 
can be used to change the design, in the following manner. 
A set, of one or more parameters is selected, say for example 
tlie time constants of the actuators and elements of the 
state weighting matri:; O. The sensitivity of the 

•.maccep^.•.^<bl e state response to these parameters is 
calculated using eqn.s (11.24) and (11.12), and the 
unacceptable response expanded in first order Taylor series 
about the nominal value of one of the parameters. This 
expansion is then used to make a change in the nominal value 
of the parameter such that the state response is improved. 
The optimal linear regulator problem is then resolved with 
the new value of the parameter. The result is an optimal 
control law which more nearly meets the state response 
criteria of the designer. 


QUALiTV 


1 
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1 1 . '1 ” L:<ton'!iion'i c«tvJ Practic3l dero^t i ons 

In the previous subsections, the idea of parameter 
sensitivity was discussed, LQ regulator theory reviewed, and 
the means of using parameter sensitivity as a redesign tool 
in optimal control theory presented. This subsection will 
discuss potential e;:tensions of the use of optimal 
sensitivity analysis, consider the practical aspects of 
computing the sensitivities, and give the reasoning befiind 
the assumptions and restrictions that were made during the 
course of the development. 

The use of optimal sensitivity analysis in other kinds 
of optimal control design processes is readily apparent, 
particulary in the ttiree areas mentioned in subsection I. a. 
The first of these was the output feedbacl: problem, where a 
number of system outputs, less than the number of states, 
are fedback. Optimal sensitivity results can be derived for 
this problem, since the necessary conditions for the optimal 
solution are Inown C73. Thus optimal sensitivity analysis 
can be used in the same manner for this problem as for the 
full state feedback problem. Perhaps more appealing is 
using the* censitivity results for the output feedback to 
design reduced ardt?r control laws. Tiiis could be done by 

f 

augment 1 rirj the system state equations with a set of dv'namics 
which repri’sents a dynamic compiensator . Treating the 



coot f i c 1 nn of tha' r.onipunijator , which define it's dynamics, 

as par ametors , and using output feedback.: theory to find 
gains for tlio states of the compensator, sensitivity 
analysis could be used to iterate on a solution for the 
compensator. fwiother passible e>:tension is to the frequency 
shaping problem of Gupta C113, where the sensitivity 
analysis could F»erhap?'i be used to define the elements of the 
frequericy dependarit we>ighting matrices. 

As far as the computation of sensitivities is 
concerned, it should be apparant that for each differ ant 
scaler paracneter os interest, a set of vector/matr i 
equations must be solved. At first, this seems like a large 
compu tat i onal burden, since these kinds of equations can be 
difficult and expensive to solve. Fortunately however, the 
coef f i c i ents of the time response equations and their 
associated sensitivity equations are exactly the same, and 
further they are independent of the parameters. This means 
ttiat once the coefficient matrices have been factored to 
solve the covariance equation for example, they need not be 
factored again. Rather, each new equation can be solved 
merely by forward and back substitution of a new 
right-hand-side. In addition, a method similiar to that 
used in El'll ho calculate gradients of cost functions may be 

f 

adapted to this analysis technique, further simplifying ttie 
C'Tfnpi.d.atiunal burden. Similarly for the absolutie time 
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rt-'&punso equd«t 1 (.'Hi* , tht? *>t.ate transsition matri:: 
con>puted only once* far Ccicfi control law. 


19 

needs to be 


Most of the assumptions and restrictions made during 
this development are related to solving the LQ regulator 
problem. The restrictions that the O' matrix be positive 
semx -def i ni ie and the R matrix be positive defitiite insure 
that a solution exists that will stabilse the system if it 
is initially unstable, if the matrix pair (A,B) is at least 


stab 1 1 i 2 ab 1 e , 

and 

the matrix 

pair 

(A,C) is 

at 1 

detect i hi e. 

The 

requi remen t 

that 

the pair 

(A, [o') 


completely observable is desirable so that a steady-state, 
symmetric; solution to the matrix Riccati equation exists. 
In 1 i gilt of the desire for a symmetric solution to the 
Riccati equation, the selection of parameters from the Q and 
R matrices sho'.'.ld be made in such a way so as to mahe sure Q 


and R are always symmetric matrices. 



» 
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In ‘af.'C 1 1 on II, the idea a-f sensitivity optimum 
solutions to problem parameters was adapted to optimal 
control problems. A derivation of optimal sensitivity 
analysis for these type problems was presented to illustrate 
that the mettiodology could be used to aid the design of 
linear fuedbacl: control laws. This section will propose a 
systematic investigation of optimal senstitivity analysis as 
design tool in optimal control design. This i nvest i gat i 'an 
has as an objective the development of a research program 
satisfying the requirements of a doctoral di ssertation . 


I I I. a - Proposed Pesearcli Program 

Section II presented the initial development of a 
design methodology based on optimal sensitivity analysis. 
Clearly tliis develofiment was incomplete in the sense that 
only the infinite time, constant coefficient linear 
regulator problem was addressed. As the first step in a 
logical development of a methodology, a broader class of 
pr‘<nl>loms inust be considered. To this end, the general 
inatheiiiat i cal formulation of optimal sensitivity analysis in 
ijpfcimal c<intr-ol t:hc^'3ry would be souight. This would include 

i 

i ■ 



An i nter cst jing cidoptati ori of optimal st-nsitivxt/ 
analysis would be in the area of reduced -order dynamic 
compensators. Or\e previous method for solving these type 
problems das been direct parameter opt i mi c at i ori of the 
contfoJ laws. Optimal sensitivity arialysis may be iibl e? to 

used for tho-.c* l;ii,ds of designs without resorting to direct 

•0 

Opt i lUi r id. i or r . To do this, the sens i i i v i t analysis would 
have to lie drivel o(ii.;d for the oi.tti.iut feedh.icl prolilern. Then, 





CRJG’NAL -- 

OF POOR QUALliV 

by r^ugiii'^n 1 1 ri'j Lhi? li/itom dyOfiinics by a canonir:i?l -form of c 
I compens.!.' t or , und s.c.d c'ct i ng as tt-ie output feodbaci; s'ariesblos 


tho state’s of 

thu (. 

:oinpensator , the 

sensitivity of 

the sys 

tern 

response to 

the 

elements of the compensator 

coul d 

be 

cal cul at'i'd . 

These 

sensitivities 

could then be 

used 

to 


rede-fine the compensator. 

As a final area of proposed research, the sensitivity 
of frequency domain roponses to parameters of the optimal 
control law formulation should bo investigated. Tltis study 
would Inelp establish a relationship between these variables 
and the frequericy t esponse of the system. Irt the course of 
this study, it may be necessary to consider the frequency 
shaping quadratic forms of Gupta till. 
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